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AESTPACT 

The pottintial of the Rasch model to deYelop scores^ 
on a ratio scale, suitable for interindivldual cornpariso from 
intact groups v;ith disparate distribution characteristics was 
investigated. The specific problems studied were- (1) the effects of 
skewed test score distributions on the ability parameter of the Rasch 
measurement model; (2) the effects of gr^up size on the ability 
parameter of the Rasch measurement model; (3) the interactive effects 
of skewed test score cSlstributions and group size on the ability 
paramater of the Rasch measurement model' and (a) the effects of skew 
and total group size on the standard errors of estimate of item log 
easiness estimates. The data for the study in the form of item 
responses were randomly selected from 120^000 students who 
participated in the Florida State-Wide Testing Program in September 
1971, The 90-item mathematics test was selected. Thirty-five raw 
score distributions characterized by seven levels of skew and five 
group sizes were constructed* Group responses were submitted to a 
computer program which estimated the model's parameters according to 
a maKlmum likelihood procedure. Results of the study indicated that 
the estimates derived from the Rasch measurement model were not , 
independent of the group used to produce them. Differences were 
minimal in the middle score range^ but large in low and high score 
range. Elevcin tables present the study data* (Author/DB) 



THE EFFECTS OF DIVERSE TEST SCOPJ DISTRIBUTION CHARACTERISTICS ' 
ON THE ESTIMATION OF THE ABILITY PARAMETER 
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QFFlCi OF iDUCATiON 
THIS DOCUMINT HAS BEEN REPRO 

- rJ^ ^ DUCID IXACTLY AS RECCIVEP FROM 

Beuiah K, Cypress the person or □bg^nization orig 

> mu^ w 1 ^^-^ 1 'r-i i * ^ ^ iNATINGiT POINTS Of VIEW OR OPIN 

^ The Florida State University* ions stated do not NECESiAmLy 

- REPRiSENT OFFICIAL OFFICE Oh EDU 

^^fc CATION POSITION OR POLICY 

^ The Rasch measuramant model has been suggested as a method 

2 of developing achievement test noms i^hich do^s not depand on the 
precision of s&mpling techniques. Due to the property of measure^ 
ment objeativity, the raw ahility scores estimated according to 
the model have baen claimed to be ralativ^ily free of the group 
used to produGe them. The potential of the Rasah model to devalop 
scores, on a ratio scale, suitable for interindividual comparisons, 
from intact groups with disparate distribution characteristics 
was investigated . 
^ The specific problems probed by the study werei 

(1) The effects of skewed test score distr^lbutione on the 
ability parameter of the Rasch measurement model\ 

(2) The effects of group size- on the ability ||arameter of 
the Rasish measur«n©nt model . 

\^(3) Vhe interacttjve effects of skewed test saore distribu- 
tions vand group siae on 'the ability parameter of the Rasch 
me a sur anient jm<^del. ... 

(4) Tiis effacta of skew knd total group size on the stand- 
ard errors of estimate of item log easiness estimates. 

A paper laresented at the Annual Meeting of the American Edueational 
Research I'issociation, N;aw Orleans, 19'73. 
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Er|c*^°^ ^l^^^^i-^^^Bii^ o£ Illinois CoUeg'^ of Medicine, Center for 
EHB^ Educational Development 
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The data for the study in the form of item responses wiire\ 

randomly selected from 120,000 itudenta who participated in tht» \ 

Florida State-Wide Testing Pk^ogram in September 1971. The 90 

item mathematiGS test was selected ^.i^ce it met the criteria 

spacified by the model for a well conrtruated achievement test 

Qpnsistlng of many it^s with a range of \asy and difficult itemB. 

Tftis was necessary to minimize standard erroVs of estimate of 

\ 

ability estimates. Thlrty»five raw score distributions charac- 

V 

teriaed by 7 levels of skew and 5 group sizes wei,e ccnatruGted 
Skew was introduced by manipulating the percentager.within aaore 

\ 

intervals of a normal distribution of 1200 freguenciek. This 
resulted in skews designated as low positive, medium positive, 
high positive, low negative, medium negative, and high negative. 
Group siEes were 1200, 600, 300, 150 and 75, Group responses 
were submitted to a computer prograii developed by Wright and 
Panehapakeaan (\^70) which estimated the model's parameters ac- 
cording to a maKinlum likelihood procedure. 

Vh& raw ability scores generated from the data of the 35 
constructed groups were compared by means of a generalized dis- 
tance function, 

d =£(Ci - gV) , 

where C represents the Rasch score in the criterion group, G is 
the Rasch score in another group, and i is the index of Raach 
score estimates for raw scores 1 to 89. The magnitude of d for 
each eomparison was expected to reveal consistent skew, size and 
interaction effects. : ^ 
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Th©, normal distribution within imch group aizeVwas the criterion 



for t\e skew effect;^^ and the largLt group aiae Vithin each iavel 
of skawWs the stan^d for the size effect. The^^ritarion for 



the intera^ive effect 'of skew and group size was the group of 

1200y norma i\y distributed test scores 

1^ investigate the effect of group size and skew on the 
standard errors of estimate of thk items, 10 easy and 10 diffi- 
cult items Qommon to all groups were selected and compared aarosa 
normally distributed groups of all siaes for the size effect, 
and across the 7 levels of skew in the groups of 1200 for the 
skew effect, 

Estiination of the sisa of the distanca function for each 
comparison revealed the following: 

(1) Wien the underlying distribution was negatively skewed, 
high Raach scores were greater than the ariterion sGores and low 
Raseh BCpres were less than criterion scores. 

(2) When the underlying distribution was positively skewed, 
high Raseh scores were less than criterion scores and low Raseh 
scores vrere greater than criterion scores. 

(3) Differences between Raseh ability scores and criterion 
scoras increased as akew became increasingly more positive or " 
negative. ■ 

Differences between scores generated from negatively 
skewed distributions and the criterion saores were larger than 

ErJc ^^^^^^^^^"^ skewed 

"^^ ^^ distributions and criterion scores » 



-4- ■ 

(5) As a group eiae deareased, high Rasch ability scores 
were greater than critirion scores and low Rasch ability scores 
were l^.ss than criterion saoras. 

(6) The size affact was not as pronounced- as the skew effect. 

(7) Standard errors of eBtimate for easy items increased 
when .he underlying distribution was negatively skewed and de- 
areased when the underlying distribution was positively skewed. 

(8) Standard errors of eatlinate for difficult items decreased 
when the underlying distribution was negatively skewed, and in- 
ereased when the underlying distribution was positively skewed. 

Results indicated that the esttaiates derived from the Rasch 
ineasurement model were not independent of the group used to pro- 
duce them. Differenees were minimal in the middle score range, 
but large in low and high score ranges. The need to calibrate 
item estimates from siiitable groups was shown by the model, but 
a paradox was revealed that incraaaed precision of calibration 
of difficult items was asiociated with negatively skewed distri- 
butions of total raw scores, but such distributions were less 
desirable for good ability estiniates . 
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VALUnS or d HELDrD m COi'PAKlKn HASCl! SCCffiB CEWERATCD 
BY (.ROUP OF 1200 BUyJIJCTS TO HASCH SCOMls GEtlERATED 
jn 'I OTliKK CmuP ULZS^B iOR 7 LKVI'M OF SKEw" 







Grouy Size- 




Average d for 
Each kvel of 
Skow 




600 


300 


150 


75 


NoMial 


4.175? 


18.2183 


23.00SO 


31.7888 


19.14972 


Low Pcisitlve 


.3251 


2,97B9 


W,B7G6 


18.14172 




Medium Positive 




1C,3D67 


'i'4.ai7e 


16.G2C0 


U.'l230 


High PoEitive 


,7020 


3,7S5JL 




32,GM03 


12,8009 


Low Wegacive 




E.8iti47 


40.3070 


28,9622+ 


19.0645 


Kedium Wegatlve 


13,7577 


14.2795 


S5,1151 


102,2770 


48,0573 


High Negative 




SC',3073 


51.7it50 


167.8923 


78,0834 
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TABLK 5 



GROUPS mmo hy haotitude or deviation 

FROM CIllTiljUOfl" GKOUP 



Rank 



Group 
Size 



Skew Laval 



Value of d 



Direction of Differences 
Low Scores High Scores 



3 
H 
S 
6 

7 
8 
3 

10 

11 

12 

13 

1!* 

15 

16 

17 

18 

£9 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 



600 
300 
ISO 
75 
75 
150 
300 
COO 
1200 
75 
ISO 
300 
600 
76 

mo 

1200 
600 
300 
.150 
300 

1200 
600 
75 
150 

1200 ■ 

. 600 

300 
1200 

ISO 

600 
75 

150 

300 
7S 



Norma J. 
Norma 1 
Normal 
Low positive 
Normal 
Low positive 
Low' positive 
Low positive 
Low positis^e 
Mediuri] positive 
Medium positive 
Medium positive 
Hediuin positive 
High poaitive 
Medium positive 
Low negative 
LoM nogative 
Low negative 
High Dositive 
High positive 
High poBltive 
High positive 
Lov/ nogative 
LoM negativQ 
Kadium negativo 
Medium riq-jntive 
Hediiin! negative 
*High negative' 
Kediurn negativfl 
nigh.nfif;ativ(3 
Madium negative 
High ne gat Ave 
High naf.ative 
High nogative 



4.1757 
10.2183 
23 1 8060 
30.8570 
31,7888 
37,2607 
'tS.M2S0 
U9.34GS 
50,3970 
73.0'I92 
75,7128. 
79,6656 
87,5504 
87.9276 
80.7310 
93,0853 
94.3236 
99.7444 
105,5441 
115.8135 
120.5734 
121,1417 
122,9866 
134,3180 
143.6725 
168,2574 
1SB.B999 
201,1395 
209.7014 
237.4C27 
246,7861 
252.7051 
257,4353 
367,7006 ■ 



<C 
<Q 
<C 

>c 
<c 
>c 
>c 
>c 
>c 
>c 
>c 
>c 
>c 
>c 

<c 
<c 
<c 
>c 
>c 
>c 
, >c 
<c 
<c 
<c 
<c 
<c 
<c 
<c 
<c 
<c 
<c 
<c 
<c 



>c 
>c 

>c 
<c 

<c 

<c 

<c 

<c 

<c 

<c 

<c 

<c 

>c 

>c 

>c 

<c 

<c 

<c 

<c 

>c 

>c 

>c. 

>c 

>c 

>Q 

>c 

>c 

>c 

>c . 

>c 

>c 



*Crit6rion^oup e 1200, normal skew lovel 
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TABLE 6- 

STANDARD BRRORS OF ESTIMATE OP LOS EASINESS ESTIMATES 
OF EASY ITEMS FOR NORHAL SK^W IN S CnOUp' SIZES 



Group Sisse 



Item 


1200 


600 


300 


ISO 


7S 


1 


yim 


.174 


,234 


,288 


,384 


2 


,100 


,W2 


,204 


.280 


^384 


•'3 


.091* 


,132 


.183 


.268 


. .483 




.090 


.134 


,187 


.268 


,334 


s 


.089 


.12? 


,183 


,257 


,334 




. .088 


.122 


,171 


.222 


,311 


7 


.085 


,125 


.178 . 


,257 


.318 


8 


.080 


,111 


.165 


,233 


*31i 


9 


,079 


• 111 


.1S4 ■ 


,219 


4S90 


10 


.079 


. ,113 


.154 


,222 


,290 
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Bi/J-'iiAiaJ ;,;Klvi.'kb Ot' LLiiiM^TL OF LOG EAMlHES£i Lsm!'ii''S 

OP uTmcuLT iTn;s. TOR mmhh nmv lu s group si:^es 



Group AivA. 
fiCO 300 ™'''~ JgJ 





,JD5 


,160 


.222 


♦ 326 


.071.1 




am 


,207 


,318 


.075 


.105 


tilts 


.211 


,318 


,07G 


,103 


.146 


.207 


.293 


.OVV 


.loa 


,165 


,23if 


,373 


.077 


,105 


,161 


,227 




,079 


,115 


.159 


.222 


.318 


,07S 


.112 


,160 


.222 


.335 




,112 


.IM 


.222 


,326 




.129 


.193 


,301 


.473 



TABLE 8 

smroARD crsRoiNS of lstimatl' of log easinms esthmtes 

OF F./iSV ITEMS TOR GROUP SIZE 1200 IH 7 LEVELS OF SKEH 



LfiVttl of Skew 



Item 


Positive 


Posit i\© 


Hii'h 
Positive 






rieQi,uin 
NegatiVo 


nlgn 

Kegative 


1 






i U 




.175 


• am 


i2Sl . 


2 


.087 


,on 


.066 


,100 


,126 


,w 


,176 


3 


*oe5 


.079 


.072 




,10U 


,108 


,120 


<♦ 


.080 


.071 


.001.1 


.090 


,101* 


,112 


,123 


5 


«077 


.069 


,062 


.080 


.116 


.133 


.147 


6 


,078 


.059 


,OCii 


,0G8 


.309 


il2tf 


.mi 


7 


.078 ■ 


,070 


.Ofil! 


.085 


,100 


.109 


ills 


8 


• .072 


.066 


,061 


.000 


,093 




.113 


9 


.069 


.063 


.061 


.079 


.107 


.132 ^ 


\^.175 


10 


.069 


.063 


loci 


,079 


,105 


,133 
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TAB.LE' 9 ■ 

STANDARD ERUOnO OF D3TIKATD OF LOG CASiKESS ESTIMATES 
OF mETICULT ITCMS FOR GROUP SISD 1200 
IN 7 LEVELS Of SKEW 



Level of 






Low 




Hish 
P<Mltivu 


* 

Noi-mal 


Low .■ 
Megativu 


Nedium 
ifegative 


■ High 
Negativa 




.'07S 


,077 


,083 


,073 


.089 . 


.068 


.066 


2 


,07S 


*on 


.075 


.074 


.072 


.070 


*067 


3 


.079 . 


.081 


,033 


,075 


.070 


.057 


,06M 


^ 


.077' 


.078 


.079 


,'076 


.07tt 


. ,073 


,063 


5 


. .oei 


,090 


,09H 


,077 


.070 


,068 


.063 


6 




.074 


,071^ 


,077 


.077 


,07S 


.072 


7 


.085 


.086 


.089 ^ ,079 


,073 


.070 


,066 


8 


.oao 


.079 


,081 


,073 


,075 


.073 


.067 


9 . 


.08S 


.007 


.092 


,080 


,073 ■ 


,074 . 


.065 


0 


,098 


.099 


,007 


,008 


,079 


,074 


.067 
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TABLE 10 



EFFICTS 



OP SKEW OH STANDARD ERRORS OF ESTIMATE 
or EASY AND DIFFICULT ITEMS 





Hefntive Skew 


Positive Bkm 


Easy Iterns 


As Skew Insrsasas 
Stanaard Errors 


As Skew Inereasas 
Standiird Brrora 
Desrgaae 


Difficult Itenis 


As Skew inereasfii 
Standard Err^TO i 
Doeraasc | 


Ab Skew tnereases 
StandardjBrrDrs 
Ineraaao 



TABLE 11 



EFFECTfl or SKEW OH STANDARD ERRORS OF ESTrMATE OF 
EASY AND DIFFICULT ITBMS ON RASCH - 
ABILITY ■SCOIUJS 





Hcgative Skow 


j 

Positive Skew 


Easy Items 


As Skew Incire^^asf 
Standard Er^^org 
InQi^ease; and- Low 

High Scores 
Increase 


As Skovf Increases 1 
Stands^ EwQi>s . 
Decreasq and Low 
Scores Ittcy'eos^i 
High SeoreB 




(A) 


CB) 


Difficult Itatn^ 


DtCMasei and Low 

High Scoroo 
Increase 


As Skew Increasisi 
Standard Errors 

; Incrt^as^ and Lw 
Scores Increase » 

■ High Saores 
Decrease 




CC) 





